If understanding uncertainty is important to good critical thinking, then so is Bayesian statistics. New evidence comes in: how does that change one's estimate of the probability that something is true? Bayes's theorem provides an answer. For a highly readable introduction to the subject, Donald A. Berry's Statistics: A Bayesian Perspective is an excellent starting point. It's a fast-paced undergraduate textbook that requires no calculus or other mathematical background. The author (formerly a professor at Duke University, now a department head at the University of Texas Medical Center) is very much first-person in his presentation. He focuses on practical examples and case studies that range widely, and entertainingly, over areas including violence on TV, malaria and sickle-cell anemia, baseball injuries, burrowing owl nests, causes of cancer, tornado forecasting, and a host of other topics.

Berry begins by introducing general concepts of probability and statistics. Along the way he presents diverse examples of good data visualization and guidelines for displaying quantitative information. The author's humorous yet critical asides are a frequent delight. For example, in Chapter 2 ("Displaying and Summarizing Data") he notes:

... If you arrange a set of groups so the bars go from low to high or high to low, the eye picks this up and associates a trend. There are several ways of displaying data and the displayer can choose the one that makes a point best. The problem is one of "multiplicities" (to be discussed in Section 3.4). There may be no way to tell a story—any story!—without deceiving. As a reader, you have to be aware of the options available to authors and make suitable adjustments. When you are the author, be aware of the power of charts and graphs and of the inevitability of deception.

And in Chapter 12, discussing the independence of samples from two populations, he observes:

... This assumption may not be perfectly apt in every problem we consider, but it holds at least approximately in all of them. Moreover, as I have indicated before, if you do not make assumptions, you will get nowhere. Make assumptions, and worry about whether they are appropriate.

Berry discusses how to gather data or design experiments to test hypotheses, and demonstrates a healthy skepticism about much published research. In Problem 12.15 dealing with "How does background music affects worker productivity?", e.g., he notes parenthetically:

I worry that the design of this study does not produce random samples from the appropriate populations and so the calculations you make are therefore themselves inappropriate. The workers in the study were not randomized to the two units. Work units are cultures in and of themselves, with work speed and "attitude" rubbing off from one worker to another. There is an aspect of this type of design for which the experimental unit is the work unit and not the worker. If that were true in the extreme, this study provides a sample of size 1 from each population of work units, ... . Conclusions are weak when the sample sizes equal 1, unless the prior information is convincing.

And in Problem 12.27, "concerning the effect on housefly lifetime of exposure to 100% oxygen":

My reservations about the design of this experiment are similar to those in the previous exercise. I do not know how many cases were used for each group, but I think there was only one. Cage variability should be assessed. Doing so requires multiple cages. Also, the effect of oxygen may depend on the number of flies in the cage, which therefore should itself be varied. Moreover, the results are sufficiently nonintuitive that there may have been a handling difference of the Temp. O₂ and control cages that would cause a reversal of the expected results. That the researchers can explain their paradoxical results may be reassuring, but it is far from convincing; the human mind can explain any observation—even those that are wrong!

Berry doesn't overlook the value of negative evidence either. In Chapter 13, "Outliers", after discussing how important samples far from the norm can be, he points out:

... This interval is inappropriately narrow because the population is not normal. So you need to be concerned about outliers even if you do not see them! Like a species of mosquito in which the males buzz but do not bite and the females do not buzz but bite—you have to worry as much when you hear nothing!

As with the dog that did nothing in the night-time, that's a widely applicable concept.

(cf. Statistics: A Bayesian Perspective, Donald A. Berry, Wadsworth Publishing Co., 1996) - ^z - 2010-08-13